
United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 

Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 



t, 



| ATTORNEY POCKET NO. | CONFIRMATION NO~ 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



09/863,296 



34610 



05/24/2001 



Sung Bae Jun 



LGE-005 



9217 



7590 



06/08/2004 



EXAMINER 



FLESHNER & KIM, LLP 
P.O. BOX 221200 
CHANTILLY, VA 20153 



LU, KUEN S 



ART UNIT 



PAPER NUMBER 



2177 

DATE MAILED: 06/08/2004 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



Office Action Summary 



Application~Nol^^~ 

09/863,296 



Examiner 

Kuen S Lu 



Applicant(s) 

JUN ETAL 



J14 



Art Unit 

2177 



The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed may reduce anv 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )S Responsive to communication(s) filed on 4/6/2004 . 
2a)KI This action is FINAL. 2b)Q This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) D Claim(s) ' is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) KI Claim(s) i^23 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)D The drawing(s) filed on is/are: a)D accepted or b)Q objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 119 

12)Q Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 

* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1) □ Notice of References Cited (PTO-892) 

2) D Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) □ Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 

Paper No(s)/Mail Date . 



4) □ Interview Summary (PTO-413) 

Paper No(s)/Mail Date. . 

5) D Notice of Informal Patent Application (PTO-1 52) 

6) □ Other: . 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 1-04) 



Office Action Summary 



Part of Paper No./Mail Date 6 



Application/Control Number: 09/863,296 Page 2 

Art Unit: 2177 

DETAILED ACTION 
Response to Amendments 

1. The Examiner has noted the Applicants' amendments made to the drawing Fig. 1 
and the claims, filed on April 6, 2004. 

In responding to Applicants' amendments made to the Claims, filed on April 6, 2004, 
the Examiner has created this Office Action for Final Rejection as shown next. 

As for the Applicants' REMARKS, filed on April 6, 2004, has been fully considered by 
the Examiner, please see discussion in the section Response to Arguments, following 
the Office Action for Final Rejection as shown next. 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth 
in section 102 of this title, if the differences between the subject matter sought to be patented and the 
prior art are such that the subject matter as a whole would have been obvious at the time the invention 
was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability 
shall not be negatived by the manner in which the invention was made. 

2. Claims 1, 3-7, 9-14 and 16-23 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Jain et al. (U.S. Patent 6360234 B1), and further in view of Ottesen 
et al. (U.S. Patent 5930493). 

As per claims 1, 10 and 18, Jain et al. (hereafter Jain) teaches the following: 
"a data server system for providing multimedia data to subscribers" at Fig. 1, elements 
130-140 combination is the data server and at Fig. 1 , element 102 and col. 3, lines 48- 
52 where a live satellite feed provides multi-media to clients; 
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"an index server system for receiving multimedia streams transferred from the data 
server system to subscribers" at Fig. 1, element 110, col. 3, lines 43-46, "extracting 
index data from the received multimedia streams" at Fig. 9, elements 510 and 530 
where Feature Extractor Framework and Metadata Track Index Manager extract the key 
frames, text and summary data, "and providing the extracted index data to subscribers" 
at Fig. 17, col. 3, lines 12-15 and at Fig. 9, elements 510 and 530 where Feature 
Extractor Framework, Metadata Track Index Manager and Output Filter Manager 
provide extracted index data to clients; and 

receiving live satellite feed at col. 3, lines 47-49, "playing the multi-media data from the 
data server system" at Fig. 1, element 140, col. 3, lines 63-67, "providing a user 
interface" at Fig. 1, element 130, col. 3, lines 53-58, "to perform an indexed search and 
browsing using the index data provided from the index server system" at col. 2, lines 12- 
15, Fig. 17, col. 3, lines 12-15. 

Jain does not specifically teach subscriber equipment for recording and subscribers, 
although Jain teaches metadata and content servers at Fig. 1, elements 130 and 140. 

However, Ottesen et al. (hereafter Ottesen) teaches multi-media server at col. 3, 
lines 56-62, real-time recording and playing at col. 8, line 64 through col. 9, line 4, and 
subscriber set-top control system and subscriber interface at col. 3, lines 51-55. 
Jain does not specifically teach subscriber equipment, although Jain does teach video 
and content servers (Fig. 1). Further, Ottesen teaches multi-media server (col. 3, lines 
56-62) and on-demand service to a large number of subscribing customers (vol. 2, lines 
14-15). 
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It would have been obvious to one having ordinary skill in the art at the time of the 
applicant's invention was made to combine Ottesen's teaching into Jain's because both 
references are devoted to multi-media distribution to the clients where fast review (Jain: 
vol. 2, lines 21-25) and efficient distribution (Ottesen: col. 3, lines 3-17) are critical to 
both Jain and Ottesen's systems and the combination of reference would have provided 
media content, keyframes temporally indexed and subscribing services simultaneously 
by an integrated multi-media system. 

Jain further teaches "index data extracted from the index server system are 
structural, semantic or summary data of the multimedia' streams include shot or scene 
data described based on temporal data" at Fig. 6, where the key frames in the key 
frame track (element 320) provides the structural data (col. 6, lines 35-47), the cc-text 
track (element 322) provides the semantic data (col. 6, lines 35-47) and the clip track 
(element 332) provides metadata, including the summary data (col. 6, lines 47-57), 
respectively. Jain further teaches time-stamping the extracted key frames for the 
purpose of correlating with the digital video or a time-code on a videotape at col. 6, lines 
30-38. 

As per claim 3, Jain teaches extracting structural index at col. 6, lines 30-38, 
semantic index at col. 6, lines 30-38 and summary index at col. 6, lines 54-57. 

Further, Jain teaches structural index data including temporally (time-stamped) key 
frames at col. 6, lines 30-38. 

As per claim 4, Jain teaches semantic index data including scene change at col. 6, 
lines 30-38. 
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As per claim 5, Jain teaches summary index data including user-defined group of 
data which includes summary index at col. 6, lines 54-57. 

As per claim 6, Jain teaches "wherein the index server system includes at least one 
indexing engine" at Fig. 4, element 111, "having a program therein for automatically 
extracting the index data" at col. 2, lines 8-15 and "an interface means for manually or 
semi-automatically extracting the index data by an operator" at Fig. 11, col. 4, lines 14- 
17. 

As per claim 7, Jain teaches "...wherein the index server system includes a 
transmitting means for transmitting the index data to the subscriber equipment " at Fig. 
1, element 112, col. 14, lines 47-49. 

As per claim 9, Jain teaches subscriber system that includes a communication 
Interface at Fig. 1 , element 150, col. 3, lines 43-47. 

As per claim 11, Jain teaches "...extracting the index data is automatically 
performed using an index engine" at Fig. 4, element 111, col. 2, lines 8-15. 

As per claim 12, Jain teaches "...the step of extracting the index data is manually 
performed by an operator" at Fig. 11, col. 4, lines 14-17." 

As per claim 13, Jain teaches "...extracting the data is semi-automatically performed 
by combining an automatic extracting system... and a manual system by an operator" at 
Fig. 4, element 111, col. 2, lines 8-15 and Fig. 11, col. 4, lines 14-17. 

As per claim 14, Ottesen teaches distributing multi-media programs concurrently to 
a plurality of subscriber set-top control systems at col. 3, lines 51-55. 

As per claim 16, Jain teaches index data extraction in real time at col. 1 , lines 66-67. 
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As per claim 17, Jain does not teach "store the multimedia stream", though Jain 
teaches "extracts the index data by indexing..." at col. 2, lines 10-15. 

However, Ottensen teaches storing multimedia streams at col. 8, lines 64 through 
col. 9, line 4. 

It would have been obvious to one having ordinary skill in the art at the time of 
the applicant's invention was made to combine Ottesen's reference into Jain's system 
by specifically recording multimedia streams at its content server such that recorded 
streams could be utilized for index extraction and then playing with indexed data 
simultaneously at later but pre-determined time which would enhance Jain's system as 
a pre-produced and pre-recorded multimedia streams provider. 

As per claim 18, Jain further teaches index extraction for multimedia streams which 
does not Specifically exclude when the streams is provided in advance at col. 2, lines 
10-15. 

As per claim 19, Ottesen teaches providing pre-recorded or pre-produced multi- 
media streams to subscribers at col. 8, lines 64 through col. 9, line 4. 

As per claim 20, Ottesen teaches providing multi-media streams to subscribers at 
the time as requested by implementing a set-top control system on an on-demand and 
pay-per-view basis. 

As per claim 21, Jain teaches "the structural data of the multimedia streams include 
shot or scene data described based on temporal data" at col. 6, lines 30-38 where key 
frames are the scene data temporally indexed by timestamps. 
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As per claim 22, Jain teaches "the semantic data of the multimedia streams include 
information on appearance or disappearance of objects, transition of background, 
occurrence and termination of event, semantic data of each section within the streams, 
and state of the object, wherein those information are described based on temporal 
Data" at Fig. 2 and col. 11, lines 11-30 and col. 6, lines 30-38 where multimedia streams 
are temporally indexes. 

As per claim 23, Jain teaches "the summary data of the multimedia streams include 
key frame or highlight data, or segment data related to summary/detail relationship or 
cause/result relationship between segments or between events, wherein such data are 
described based on temporal data" at Figs. 2, 6 and 9, and col. 6, lines 30-38 and 48-57 
where key frames are temporally indexed by timestamps and relationship between key 
frames and data from other tracks is shown,. 

3. Claims 8 and 15 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Jain et al. (U.S. Patent 6360234 B1) in view of Ottesen et al. (U.S. Patent 
5930493), as applied to claims 1, 10 and 18-20, and further in view of Aras et al.(U.S. 
Patent 5872588). 

As per claims 8 and 1 5, Jain or Ottesen does not specifically teach encoder or 
decoder as described in "...an encoder that encodes the index data to provide only 
permitted users with the index data, and wherein the subscriber equipment includes a 
decoder that decodes the index data received from the index server system", though 
Jain teaches index server system on extracting index data at col. 3, lines 43-46 and 
Fig. 9, col. 8, lines 23-32, and transmitting index data at col. 14, lines 47-49. 
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However, Aras teaches decoding at col. 24, lines 44-51 and encoding at Section 
"AVI Encoding Mechanism", col. 11, line 43. 

It would have been obvious to one having ordinary skill in the art at the time of the 
applicant's invention was made to combine Ottesen and Aras' references into Jain's 
teaching by implementing encoding and decoding functions to Jain's cataloger system 
because without such an implementation, subscription of indexed data from the server 
would not be feasible, and thus the commercial potential of the index server system 
would not be developed. 

Conclusion 

4. The prior art made of record 

A. U.S. Patent No. 6360234 

B. U.S. Patent No. 5930493 

C. U.S. Patent No. 5872588 

The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

D. U.S. Patent No. 6018744 

E. U.S. Pub. No. 2002/0146233 A1 

F. U.S. Patent No. 5802283 

G. U.S. Pub. No. 2002/0129140 A1 

H. U.S. Patent No. 5625404 

I. U.S. Patent No. 5483276 

J. U.S. Pub. No. 2002/0170062 A1 
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K. U.S. Pub. No. 2002/0161747 A1 

L. U.S. Patent 5790176 

Response to Arguments 
5. The Applicants' arguments filed on April 6, 2004 have been fully considered, but they 
are not persuasive, for the Examiner's response, please see discussion below. 

a) . At Page 9, Claim 1 , Applicants argued "Jain and Ottesen do not teach or suggest 
... index server system extracting index data from ... wherein index data are structural, 
semantic and summary data ... based on temporal and combinations thereof 

As to the above argument a), the Examiner disagreed because Jain teaches index 
service by using Feature Framework to process metadata from Feature Extractors (col. 
8, lines 27-32) which extracts key frames, text, among others and, Jain further teaches 
time-stamping the key frames for the purpose of correlating with the digital video or a 
time-code on a videotape. Further, Jain teaches structural, semantic and summary data 
at Fig. 6, where the key frames in the key frame track (element 320) provides the 
structural data (col. 6, lines 35-47), the cc-text track (element 322) provides the 
semantic data (col. 6, lines 35-47) and the clip track (element 332) provides metadata, 
including the summary data (col. 6, lines 47-57), respectively. 

b) . At Page 10, Claims 2-5, Applicants further argued "Although Jain discloses 
tracking metadata types, ... such meta data types are not provided as index data". 

As to the above argument b), the Examiner disagreed because Jain does teach 
extracting key frames, cc-text and metadata summary as the structural, semantic and 
summary data, as described in last section as previously described in response to item 
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a). Further, Jain teaches time-stamping the key frames for the purpose of correlating 
with the digital video or a time-code on a videotape at col. 6, lines 30-38. Thus, the key 
frames extracted with data based on time-stamp is temporally indexed. 

c). At Page 1 1 , Claim 1 , Applicants submitted that "... it would be obvious to combine 
Ottesen into Jain teaching..." an improper hindsight reasoning. 

In response to Applicants' argument that there is no suggestion to combine the 
references, the examiner recognizes that obviousness can only be established by 
combining or modifying the teachings of the prior art to produce the claimed invention 
where there is some teaching, suggestion, or motivation to do so found either in the 
references themselves or in the knowledge generally available to one of ordinary skill in 
the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988) and In re 
Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992). In this case, Jain does not 
specifically teach subscriber equipment, although Jain does teach video and content 
servers (Fig. 1). Further, Ottesen teaches multi-media server (col. 3, lines 56-62) and 
on-demand service to a large number of subscribing customers (vol. 2, lines 14-15). It 
would have been obvious to one having ordinary skill in the art at the time of the 
applicant's invention was made to combine Ottesen's teaching into Jain's because both 
references are devoted to multi-media distribution to the clients where fast review (Jain: 
vol. 2, lines 21-25) and efficient distribution (Ottesen: col. 3, lines 3-17) are critical to 
both Jain and Ottesen's systems and the combination of reference would have provided 
media content, key frames with temporal indexed data and subscribing services 
simultaneously by an integrated multi-media system. 
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d) . At Page 1 1 , Claim 1 , Applicants further submitted "Ottesen does not teach or 
suggest any indexing of multimedia... Further Ottesen teaches away from a video 
cataloger 110 in Jain by disclosing 

As to the above argument e), the Examiner disagreed because Ottesen is referenced 
for teaching on a combined multimedia server and subscriber feature, not for its 
teaching on indexing of multimedia. As described in the response to item a), the 
Examiner has established that Jain teaches indexing service of multimedia and 
providing structural, semantic and summary data, teaching the feature of cataloger does 
not teach away indexing of multimedia. 

e) . At Page 12, Claims 8 and 15, Applicants further argued "Aras does not teach or 
suggest at least features of index server system 

As to the above argument e), the Examiner disagreed because Aras is referenced for 
teaching encoding, not for its teaching on indexing of multimedia. As described in the 
response to item a), the Examiner has established that Jain teaches indexing service of 
multimedia and providing structural, semantic and summary data, teaching the feature 
of cataloger does not teach away indexing of multimedia. 

As to dependent claims 3-7. 9. 11-14 and 16-23 . which depend on claims 1 and 10, 
respectively, the Examiner applies the above stated arguments for the respective claim 
upon which they depend. 

6. In light of the forgoing arguments, the U.S.C 103 rejection for Claims 1 , 3-23 are 
hereby sustained. 

Conclusions 
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7. THIS ACTION IS MADE FINAL. 

The Applicants are reminded of the extension of time policy as set forth in 37 CFR 
1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

8. The prior art made of record, listed on form PTO-892, and not relied upon, if any, is 
considered pertinent to applicant's disclosure. 

If a reference indicated as being mailed on PTO-FORM 892 has not been enclosed in 
this action, please contact Lisa Craney whose telephone number is (703) 305-9601 for 
faster service. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Kuen S Lu whose telephone number is 703-305-4894. 
The examiner can normally be reached on 8 AM to 5 PM, Monday through Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Breene can be reached on 703-305-9790. The fax phone number for 
the organization where this application or proceeding is assigned is (703) 872-9306. 
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Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is 703-305- 
3900. 
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